Make the deferred header flush grace configurable - #1795
Conversation
The SEP-2575 HTTP status mapping only works while the response headers are still uncommitted. StreamableHttpPostTransport bounded that wait at a fixed 250 ms, so a dispatch slower than the window committed a default 200 and a later JSON-RPC error rode the committed status. That made the mapped status a function of machine scheduling rather than of server behavior. Expose the window as DeferredHeaderFlushGrace on StreamableHttpServerTransport and HttpServerTransportOptions, defaulting to the historical 250 ms so the out-of-the-box behavior is unchanged. Timeout.InfiniteTimeSpan never forces the flush, which makes the mapping deterministic. The raw HTTP conformance tests now disable the bound, and two tests pin both halves of the tradeoff against a handler that runs past the old window.
|
Thanks for tracking this down and proving the 250 ms race. I don't think we should make the grace configurable. The production default would remain nondeterministic, while an infinite grace just trades the race for waiting until application work produces a response. We're going to remove the grace instead. For 2026-07-28 requests, we'll wait for the first JSON-RPC message so the SEP-2575 status mapping is deterministic. Older protocol revisions will keep the eager header flush. A long-running handler can emit a real progress notification when the client requested progress, but we don't want to add a fake notification or a Streamable HTTP-specific I'm going to close this in favor of that approach. Thanks again for the investigation and tests. |
|
That makes sense, and I think your reasoning is better than mine. Making the grace configurable leaves the default nondeterministic, which is the actual complaint, so it moves the problem to whoever reads the docs rather than solving it. Waiting for the first JSON-RPC message on 2026-07-28 and keeping the eager flush for older revisions gets determinism without a new knob. One thing you may want out of this before it disappears. The conformance test that pins the race does not depend on the option, so it should carry over to your approach unchanged:
The other one, Happy to leave it here, just did not want the first test to get closed along with the design it was attached to. Thanks for the detailed reasoning, and for saying what the replacement is rather than only what was wrong. |
2 participants
Closes #1772.
The SEP-2575 status mapping only holds while the response headers are still uncommitted, and
StreamableHttpPostTransportbounded that wait at a fixed 250 ms. A dispatch slower than that window commits a default 200, so theMissingRequiredClientCapabilityerror arriving afterwards rides the committed status, which is what makesJuly2026Post_MissingRequiredCapability_Returns400depend on machine load rather than on server behavior. This change exposes the window asDeferredHeaderFlushGraceonStreamableHttpServerTransportandHttpServerTransportOptions, keeping the historical 250 ms as the default so nothing changes out of the box, andTimeout.InfiniteTimeSpandisables the bound so the mapping becomes deterministic.The raw HTTP conformance tests now disable the bound so they assert server behavior instead of scheduling, and two new tests pin both halves of the tradeoff against a handler that deliberately runs past the old window. I confirmed the direction of the bug by pinning the grace back to 250 ms against that slow handler and watching the error come back over a committed 200.
Happy to take this a different direction if you would rather drop the bound outright on the JSON response path or solve it purely on the test side.